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^ ■ of superstring theory with fluxes and is expected to lead to a deeper understanding of 
mirror symmetry. Guahieri's notion of generalized complex submanifold seems to be a 
^ . natural candidate for the description of branes in this context. Recently, we introduced a 
' Batalin-Vilkovisky fleld theoretic realization of generalized complex geometry, the Hitchin 
sigma model, extending the well known Poisson sigma model. In this paper, exploiting 
Gualtieri's formalism, we incorporate branes into the model. A detailed study of the 
boundary conditions obeyed by the world sheet fields is provided. Finally, it is found that, 
when branes are present, the classical Batalin-Vilkovisky cohomology contains an extra 
sector that is related non trivially to a novel cohomology associated with the branes as 
generalized complex submanifolds. 



Hitchin's generalized complex geometry has been shown to be relevant in compactifications 
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1. Introduction 



Mirror symmetry is a duality relating compactifications of type IIA and type IIB 
superstring theory, which yield the same four-dimensional low energy effective theory. It 
has played an important role in the study of Calabi-Yau compactifications for both its 
theoretical implications and practical usefulness. Recently, more general compactifications 
allowing for non Ricci-flat metrics and NSNS and RR flirxes have been object of intense 
inquiry. The natural question arises about whether mirror symmetry generalizes to this 
broader class of compactifications and, if so, which its properties are. This program was 
outlined originally in refs. [1,2] and was subsequently implemented with an increasing 
degree of generality in a series of papers [3-5]. These studies indicate that mirror symmetry 
can be defined for a class of manifolds with SU(3) structure. In spite of these advancements, 
certain aspects of mirror symmetry remain mysterious. 

In 2002, Hitchin formulated the notion of generalized complex geometry, which at the 
same time unifies and extends the customary notions of complex and symplectic geometry 
and incorporates a natural generalization of Calabi-Yau geometry [6]. Hitchin's ideas 
were developed by Gualtieri [7], who also worked out the theory of generalized Kaehler 
geometry. The SU(3) structure manifolds considered in [4,5] are generalized Calabi-Yau 
manifolds as defined by Hitchin. This indicates that generalized complex geometry may 
provide the mathematical set up appropriate for the study of mirror symmetry for general 
fiux compactifications. 

Type II superstring Calabi-Yau compactifications are described by (2,2) supercon- 
formal sigma models with Calabi-Yau target manifolds. These field theories are however 
rather complicated and, so, they are difficult to study. In 1988, Witten showed that a 
(2, 2) supersymmetric sigma model on a Calabi-Yau space could be twisted in two differ- 
ent ways, to give the so called A and B topological sigma models [8]. Unlike the original 
untwisted sigma model, the topological models are soluble: the calculation of obscrvables 
can be reduced to classical problems of geometry. For this reason, the topological models 
constitute an ideal field theoretic ground for the study of mirror symmetry. 

Mirror symmetry relates the A and B models with mirror target manifolds [9]. The A 
and B models depend only on the symplectic and complex geometry of the target manifold, 
respectively. Therefore, generalized complex geometry, which unifies these two types of 
geometry, may provide a natural mathematical framework for a unified understanding of 
them. It is also conceivable that topological sigma models with generalized Kaehler targets 
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may exhibit the form of generahzed mirror symmetry encountered in flux compactifications 
of superstring theory [10-15]. 

D-branes are extended sohtonic objects of the superstring spectrum, which are ex- 
pected to play an important role in the ultimate non perturbative understanding of super- 
string physics. They appear as hypersurfaces in space-time where the ends of open strings 
are constrained to lie. D-branes appear in a (2, 2) superconformal sigma model as certain 
boundary conditions for the world sheet fields. Topological branes appear in the associated 
topological sigma models again as boundary conditions. The branes of the A and B model 
are called A- and B-branes, respectively. Expectedly, mirror symmetry exchanges the sets 
of A-branes and B-branes of mirror manifolds. This constitutes a major motivation for 
their study. 

In 1994, Kontsevich formulated the homological mirror symmetry conjecture [16], 
which interprets mirror symmetry as the equivalence of two triangulated categories: the 
bounded derived category of coherent sheaves and the derived Fukaya category of graded 
Lagrangian submanifolds carrying flat vector bundles. After the emergence of D-branes in 
superstring theory, it was argued that the topological B-branes formed a category which 
could be identifled with the derived category of coherent sheaves [17,18]. Since mirror 
symmetry exchanges A-branes and B-branes, one would expect that the topological A- 
branes also form a category and that this category could be identified with the derived 
Fukaya category. This belief was supported by Witten's original work [19], where A- 
branes appeared as Lagrangian submanifolds, and by the analysis of ref. [20], which 
showed that ghost number anomaly cancellation required these Lagrangian submanifolds 
to be graded. However, other studies indicated that there could be A-branes which were 
not Lagrangian submanifolds [21]. The careful analysis of [22] showed that a class of 
coisotropic submanifolds carrying non trivial line bundles could also serve as A-branes, at 
least at the classical level. This finding suggests that the category of A-branes should be 
an appropriate extension of the derived Fukaya category including the coisotropic branes. 

For reasons explained above, it is expected that generalized complex geometry may 
furnish the appropriate framework for the understanding of mirror symmetry also in the 
presence of branes. In this case, the notion of generalized complex submanifold formulated 
by Gualtieri in [7] should play an important role. In fact, it includes as particular cases 
all known examples of topological branes, including the coisotropic ones. However, to the 
best of our knowledge, not much has been tried in this direction so far [10,15,23]. 

Any attempt to understand mirror symmetry in the light of generalized complex ge- 
ometry unavoidably must go through some sigma model realization of this latter. Several 
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such realizations have been proposed so far [10,13,23-32]. They have remarkable proper- 
ties and are also interesting in their own. This paper is a further step along this line of 
development, as we shall describe next. 

To a varying degree, the sigma model realizations of generalized complex geometry 
are all related to the well-known Poisson sigma model [33,34]. This is not surprising, in 
view of the fact that a generalized complex manifold is also a Poisson manifold. In [31], 
adapting and generalizing the formulation of the Poisson sigma model of ref. [35], based 
on the Batalin-Vilkovisky quantization algorithm [36,37], we formulated a new realization 
called Hitchin sigma model. We showed that the algebraic properties and integrability 
conditions of the target space generalized almost complex structure are sufficient, non 
necessary conditions for the fulffilment of the Batalin-Vilkovisky classical master equation 
(S*, S) = 0. Further, a non trivial relation between a sector of the classical Batalin- 
Vilkovisky cohomology, on one hand, and a generalized Dolbeault cohomology and the 
cohomology of the generalized deformation complex, on the other, was found. 

In this paper, we continue the study of Hitchin sigma model on a generalized complex 
manifold and show how generalized branes can be incorporated into it. We argue that 
branes are aptly described by Gualtieri's theory of generalized complex submanifolds [7]. 
In the presence of a brane, the world sheet has a non empty boundary. The fields therefore 
must obey appropriate boundary conditions, which are determined and analyzed in detail 
in the paper. We show that the algebraic properties and integrability conditions of the 
target space generalized almost complex structure as well as the algebraic properties of 
the brane as generalized complex submanifold are sufficient, non necessary conditions 
for the fulfillment of the Batalin-Vilkovisky classical master equation {S, S) = 0. The 
compatibility of the boundary conditions with the so called b symmetry and with the 
overall classical Batalin-Vilkovisky cohomological structure is ascertained. Further, we 
find that, when branes are present, the Batalin-Vilkovisky cohomology contains an extra 
sector that is related non trivially to an hitherto unknown cohomology associated with 
the branes as generalized complex submanifolds. Finally, we compare our approach with 
the Alexandrov-Kontsevich-Schwartz-Zaboronsky formulation of the Poisson sigma model 
without and with branes worked out in refs. [38,39]. 

The plan of this paper is as follows. In sect. 2, we review the basic notions of 
twisted generalized complex geometry. In sect. 3, we illustrate the theory of generalized 
complex submanifolds and show its use for the description of generalized branes. In sect. 
4, we introduce the 2-dimensional de Rham superfield formalism for world sheets with 
boundary. In sect. 5, we review the twisted Hitchin sigma model in the absence of branes 
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and then show how branes can be incorporated by imposing suitable boundary conditions 
on the fields. In sect. 6, we analyze the compatibility of the boundary conditions with 
the Batalin-Vilkovisky cohomological structure when branes are present and describe in 
detail the relation between the Batalin-Vilkovisky cohomology and the generalized complex 
submanifold cohomology of the branes. Finally, in sect. 7, we compare our results with 
related works on the Poisson sigma model with coisotropic branes. 

2. Generalized complex geometry 

The notion of generalized complex structure was introduced by Hitchin in [6] and 
developed by Gualtieri [7] in his thesis. It encompasses the usual notions of complex and 
symplectic structure as special cases. It is the complex counterpart of the notion of Dirac 
structure, introduced by Courant and Weinstein, which unifies Poisson and symplectic 
geometry [40,41]. In this section, we review the basic definitions and results of generalized 
complex geometry used in the sequel of the paper. 

Let M be a manifold. In what follows, M will always be of even dimension d, since 
the basic structures of generalized complex geometry exist only in this case. 

The constructions of generalized complex geometry are all based on the vector bundle 
TM © T*M. A generic section X + ^ e C°°{TM © T*M) of this bundle is the direct sum 
of a vector field X e C^{TM) and a 1-form ^ G C^{T*M). 

TM Q)T*M is equipped with a natural indefinite metric of signature {d, d) defined by 



for X + F + ?7 e C°°(TM®r*M), where iy denotes contraction with respect to a vector 
field V. This metric has a large isometry group. This contains the full diffeomorphism 
group of M, acting by pull-back. It also contains the following distinguished isometrics, 
called h transforms, defined by 



where h e C°°(A^T*M) is a 2-form. As it turns out, h transformation is the most basic 
symmetry of generalized complex geometry. 



(X + ^,F + r7) = i(ixr7 + irO, 



(2.1) 



exp(6)(X + 0=X + e + zxfe, 



(2.2) 



An H field is a closed 3-form H e C°°{A^T*M): 



dmH = 0, 



(2.3) 
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where cIm is the exterior differential of M. ^ H measures the so called twisting. The 
pair (M , H) is called a twisted manifold. (M , H) is characterized by the cohomology class 
[H] e iy^(M, IR). By definition, b transformation shifts H by the exact 3-form —dMb: 

H' = H- dMb. (2.4) 

So, the cohomology class [H] is invariant. 

On a twisted manifold (M, H), there is a natural bilinear pairing defined on C°°(TM© 
T*M) extending the customary Lie pairing on C°°{TM), called H twisted Courant brack- 
ets [40,41]. It is given by the expression 

[X + C,Y + 7j]h^ [X, Y] + Ixv - IyC - \dM{ixr] - iyi) + ixiyH, (2.5) 

with X -\-^,Y -\--q e C°°{TM © T*M), where ly denotes Lie derivation with respect to 
a vector field V. The pairing is antisymmetric, but it fails to satisfy the Jacobi identity. 
However, remarkably, the Jacobi identity is satisfied when restricting to sections X + 
^,Y + r] e C°°{L), where L is a subbundle of TM ®T*M isotropic with respect to ( ,) 
and involutive (closed) under The brackets [^]h are covariant under the action of 

the diffeomorphism group. They are also covariant under b transform, 

[exp(6)(X + e), exp(6)(y + r/)]^^ = exp(6)[X + C, 1^ + v]h'. (2.6) 

where H' is the h transform of H given by (2.4). 

A generalized almost complex structure J7 is a section of (End (TM © T*M)), 
which is an isometry of the metric ( , ) and satisfies 

= -1. (2.7) 

The pair (M, J) is called a generalized almost complex manifold. The group of isometrics 
of ( , ) acts on by conjugation. In particular, the b transform of is given by 

J' = exp(-6)Jexp(6). (2.8) 

If M is equipped with an H field and a generalized almost complex structure JT, the 
triple (M, H, J') is called a twisted generalized almost complex manifold. The generalized 
almost complex structure J is H integrable if its ±^/^ eigenbundles are involutive with 



The sign convention of the H field used in this paper is opposite to that of ref. [7] . 
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respect to the twisted Courant brackets [,]h- ^ In that case, J is called an H twisted 
generalized complex structure. The triple (M, H, J) is then called a twisted generalized 
complex manifold. It can be shown that this condition is equivalent to the vanishing of the 
appropriate H twisted generalized Nijenhuis tensor 

NH{X + C,Y + rj) = 0, (2.9) 

for all X + ^,Y + r] e C°°{TM ® T*M), where 

NniX + ^,Y + ri) = [X + C,Y + rj]H + J[J{X + e), Y + r/]^ (2.10) 

+ J[X + J{Y + v)]h- [J{X + 0, J{Y + v)]h. 

H integrability is preserved by h transformation: if J" is an twisted generalized complex 
structure and H' and J' are the h transform of H and J, respectively, then J' is an H' 
twisted generalized complex structure. 

In the untwisted formulation of generalized complex geometry, the H field vanishes 
throughout. To preserve the condition H = 0, b transformation must be restricted. By 
(2.4), it follows that only closed b fields are allowed, dub = 0. This restriction is not 
necessary in the twisted case, which is the one we deal with mostly in this paper. 

In practice, it is convenient to decompose a generalized almost complex structure 
in block matrix form as follows 




(2.11) 



where J e C^{TM ® T*M), P e C^{A^TM), Q e C°°(a2T*M) and express all the 
properties of J, in particular its integrability, in terms of the blocks J, P, Q. 

For later use, we write in explicit tensor notation the conditions obeyed by the fields 
H, J, P, Q. 

An H field satisfies the closedness equation 

daHbcd — dbHacd + dcHabd — ddHabc — 0, (2-12) 
(cf. eq. (2.3)). Under b transform, we have 

H'abc = Habc — {dJ>bc + db^ca + dc^ab) , (2.13) 



The ±-\/^ eigenbundles of J are complex and, thus, their analysis requires com- 
plexifying TM ®T*M leading to (TM ® T*M) ® C. 
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(cf. eq. (2.4)). 

If J" is a generalized almost complex structure, the tensors J, P, Q satisfy 

pab ^ pba ^ (2.14a) 

Qab + Qba^O, (2.146) 

J\rb + P""Qc6 + S\ = 0, (2.15a) 

J«^pc& ^ Jb^pca ^ (2.156) 

QacJ''b + QbcJ''a = 0, (2.15c) 

on account of (2.7). Upon using (2.2), (2.8), we find that, under b transform, 

p/a& ^ pab^ (2.16a) 

= J\ - P^'bab, (2.166) 

Q'ab = Qab + bacJ'b - + P'Xabdb- (2.16c) 

The H integrability condition (2.9) of a generalized almost complex structure can 
be cast in the form of a set of four tensorial equations 

Ah'''''' = 0, (2.17a) 

BHa'^ = 0, (2.176) 

CHab'' = 0, (2.17c) 

DHabc = 0, (2.17d) 

where Ah, Bh, Ch, Dh are the tensors defined by 

^^abc ^ padQ^pbc ^ pbdQ^pca ^ pcdg^pab ^ (2.18a) 

BHa'" = J'^addP'' + P'^'idard - ddJ" a) - P'^daJ^ - ddj\) (2.186) 

-a„(J^P'^'=) + P'"^P-iyade, 

Cnab'^ = J^addJ'^b — J'^bddJ'^a — J'^ddaJ'^b + J'^ddbJ'^a (2.18c) 

+ P'''^{ddQab + daQbd + dbQda) — J'^ aP'^'^ Hbde + J'^bP'^'^ Hade, 

9 



Dnahc = JaiddQbc + dbQcd + dcQdb) + JbiddQca + dcQad + OaQdc) {2.18d) 
+ J'^ciddQab + daQbd + dbQda) " da{QbdJ'^ c) " db{QcdJ'^a) " ^ciQadJ'^b) 
— Habc + J'^aJ^'bHcde + J'^bJ^'cHade + J'^ cJ"" aHbde- 

The above expressions were first derived in a different but equivalent form in [25] and 
subsequently in the form given here in [31]. 

One of the most interesting features of generalized complex geometry is its capability 
for a unified treatment of complex and symplectic geometry, as we show below. However, 
as noticed by Hitchin, these geometries do not exhaust the scope of generalized complex 
geometry. In fact, there are manifolds which cannot support any complex or symplectic 
structures, but do admit generalized complex structures [6]. These facts explain the rea- 
son why Hitchin's construction is interesting and worthwhile pursuing and not simply an 
elegant repackaging of known notions. 

Complex geometry can be formulated in terms of generalized almost complex struc- 
tures J of the form 

J-[i _°.), (2.19) 

where J is an ordinary almost complex structures i.e J'^cJ^b = —S°'b- J satisfies (2.17a-(i) 
if J, E. satisfy 

J'^addJ\ - J'^bddJ\ - J''ddaJ\ + J'^ddbJ'^a = 0, (2.20) 

Habc — J'^aJ'^bHcde — J'^bJ'^cHade — J'^cJ'^aHbde = 0. (2.21) 

Eq. (2.20) is nothing but the statement of the vanishing of the Nijenhuis tensor of J and, 
so, it implies that J is a complex structure. Eq. (2.21) states in turn the 3-form H is of 
type (2, 1) + (1, 2) with respect to J. 

Similarly, symplectic geometry can be formulated in terms of generalized almost com- 
plex structures J' of the form 

J={1 -'?"'). (2.22) 
where Q is pointwise non singular a 2-form. J satisfies {2.17 a-d) ii Q, H satisfy 

daQbc + dbQca + dcQab = 0, (2.23) 
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Habc = 0. (2.24) 

Eq. (2.23) states that the 2-form Q is closed and, so, it imphes that Q is a syniplectic 
structure. By eq. (2.24), the H field necessarily vanishes in the syniplectic case. 

In the Hitchin sigma model studied in [31], the action S contains a topological Wess- 
Zumino term defined up to the periods of the closed 3-form H. In the quantum path 
integral, so, the weight exp(V— l^") is unambiguously defined only if H/2n has integer 
periods. Therefore, the cohomology class [H/2n] belongs to the image of i7'^(M, Z) in 
H^{M,'R). This case is particular important for its relation to gerbes. In this context, 
the b tansforms with b a closed 2-form such that [b/2'K] is contained in the image of 
H'^{Mj Z) in H'^{M, M) represent the gerbe generalization of gauge transformations. 

In [43], it was shown that a sigma model on a manifold M with NSNS background 
H has (2,2) supersymmetry if the twisted manifold {M,H) is "Kaehler with torsion". 
This means that M is equipped with a Riemannian metric g and two generally different 
complex structures J± such that g is Hermitian with respect to both J± and that J± are 
parallel with respect to two different metric connections V± with torsion proportional to 
±iJ. The presence of torsion implies that the geometry is not Kaehler. As shown in [7], 
these geometrical data can be assembled in a pair of commuting H twisted generalized 
complex structures jTi, i = 1, 2, describing a generalized Kaehler geometry. 

The (2, 2) supersymmetric sigma model has been studied mostly for H = and J+ = 
J_, when M is a true Kaehler manifold. In this case, the generalized complex structures J7i 
J'2 are of the special form (2.19), (2.22) and encode the complex and symplectic geometry 
of M, respectively. The associated A and B topological sigma models depend on only one 
of these, depending on the topological twisting used [8]: the A model depends only on J'2, 
the B model only on Ji . A topological sigma model for generalized Kaehler geometry has 
been proposed in [13]. The Hitchin sigma model [31], which is the main topic of this paper, 
can be defined for a general twisted generalized complex structure J not necessarily of the 
form of those appearing in generalized Kaehler geometry. 

3. Generalized complex submanifolds and branes 

According to [7], a generalized submanifold of a twisted manifold (M, H) (cf. sect. 2) 
is a pair (W, F), where W is a submanifold of M and F e C°^{a'^T*W) is such that 

Lw*H = dwF, (3.1) 



^ See [42] for background material about this topic. 
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where tw '■ W ^ M is the natural injection and dw is the differential of W. Note that, 
by (2.3) and (3.1), the pair {H,F) is a representative of a relative cohomology class in 
H^{M,W,R). By (2.4), (3.1), F transforms non trivially under a b transform, viz 

F' = F- iw*b. (3.2) 

where h G C^{h^T*M). In this way, if (W,F) is a generalized submanifold of {M,H), 
then (VF, F') is a generalized submanifold of {M,H'), where H' , F' are the b transforms 
of H, F given by (2.4), (3.2). 

The generahzed tangent bundle T^W of {W, F) is the subbundle of TM ®T*M\w 
spanned by the restriction to W of those sections X + ^ e C°°{TM © T*M) such that 
X\w e C^iTW) and that 

Lw*^ = ixiw^- (3-3) 

(Here, TW is viewed as a subbundle of TM\w.) It is easy to see that b transform acts 
naturally on T^W. Indeed, one has exp{b)T^'W = T^W. In fact, this is the main reason 
why T^W is defined the way indicated above. 

Suppose {M,H,J) is a twisted generalized almost complex manifold (cf. sect. 2). A 
generalized submanifold (VF, F) of (M, H) is a generalized almost complex submanifold of 
(M, H, J), if T^W is stable under the action of J seen here as a section of C°°(End (TM© 
T*M)). When J' is H integrable and, therefore, (M, H, J') is a twisted generalized complex 
manifold, we call {W,F) a generalized complex submanifold of {M,H,J). ^ This notion 
is covariant under b transformation: if (IF, F) is a generalized almost complex submanifold 
of {M,H,J'), then {W,F') is a generalized almost complex submanifold of (M, H' , J''), 
where H', J' , F' are the b transforms of H, J, F given by (2.4), (2.8), (3.2). 

As is well known, at each point of IF, there are coordinates of M = (t*,t'') such 
that, locally, the submanifold IF is described by the equation 1^ = and coordinatized 
by the t\ We call such coordinates adapted. Here, middle Latin indices i, j, ... take the 
values 1, . . . , dim IF, while late Greek indices p, a, ... take the values 1, . . . ,d — dim IF. 
The abstract geometrical notions outlined above are conveniently expressed in terms of 
adapted coordinates. The relations so obtained are adapted covariant, that is they have 
the same mathematical form for all choices of adapted coordinates. The tensor components 



^ A generalization of Gualtieri's definition of generalized complex submanifold can be 
found in ref. [44]. 
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Habci P°'^i J°'b Qab of the H field and the blocks P, J, Q of the generalized almost complex 
structure J (cf. eq. (2.11) entering in them are tacitly assumed to be restricted to W to 
avoid the cumbersome repetition of |^. Since the F field is defined only on W anyway, no 
restriction of the components is involved. 

Relation (3.1), connecting H and F reads simply 

Hijk = diFjk + djFki + dkFij (3.4) 
at W. Under a b transform, one has 

F'ij = Fij — bij, (3.5) 

as follows from (3.2). 

If X + ^ e C°°{TM © T*M) restricts to a section of the generalized tangent space 
T^W, then one has 

XP = 0, (3.6a) 
+ FijX^ = (3.66) 

at W, as follows from (3.3). 

If (M, H, J) is a twisted generalized almost complex manifold and (W, F) is a gener- 
alized almost complex submanifold of (M, i?, J"), then, in the block representation (2.11) 
of J", one has 

P^'" = 0, (3.7a) 
jP. _ pPip^^ ^ 0, (3.76) 

Qij — J^iFjk + J^jFif; + P^^Fj^iFij = (3.7c) 

at W . These relations follow from (3.6a, 6) and imposing the stability of T^W under the 
action of J . It is straightforward to verify that these conditions are compatible with H 
integr ability conditions {2.17 a-d). 

Suppose that J is aiv H twisted generalized complex structure of the form (2.19). 
Then, J, H satisfy (2.20), (2.21) and, so, J is a complex structure and H is (2, 1) + (1,2) 
3-form. In this case, eqs. (3.7a-c) become simply 

JPi = 0, (3.8a) 
j\Fkj + J^Fife = (3.86) 
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at W. Eq. (2.20), (3.8a) entails that the J^j are the components of a complex structure 
Jw on W, which is therefore a complex submanifold of M. Eq. (3.86) entails in turn that 
F is (1,1) 2-form of W. 

Suppose that J is an H twisted generalized complex structure of the form (2.22). 
Then, Q, H satisfy (2.23), (2.24) and, so, Q is a symplectic structure and H vanishes. 
Then, eqs. (3.7a-c) become simply 

Q-^^'" = 0, (3.9a) 
Q-^P^Fji = 0, (3.96) 
Qij - Q-^'^'Fk.Fi^ = (3.9c) 

at W. Further, as = 0, (3.4) furnishes 

diFjk + djFki + dkFij = 0. (3.10) 

Eq. (3.9a) entails that maps the conormal bundle N*W into the tangent bundle TW 
oiW. This means that, by definition, that W is coisotropic. Condition (3.10) entails that F 
is a closed 2-form. The remaining conditions (3.96, c) do not have any simple geometrical 
interpretation. (See however [7] for an attempt in this direction.) The interpretation of 
the geometry of W simplifies if one adds by hand the condition 

Qij = (3.11) 

at W, in virtue of which (3.96, c) are automatically fulfilled, as is easy to see. Eq. (3.11) 
entails that Q maps the tangent bundle TW into the conormal bundle N*W of W. Hence, 
when (3.7a) and (3.11) are satisfied, the submanifold W is simultaneously isotropic and 
coisotropic, hence Lagrangian. This means that, by definition, that W is isotropic. (3.11) 
further entails the vanishing of the F field 

F,j = 0. (3.12) 

In this paper, we shall provide fresh evidence in favor of the claim that, in a consistent 
sigma model on a twisted generalized complex manifold (M, H, J) , branes should be gen- 
eralized complex submanifolds (W, F) [10,15,23]. We note that the formalism expounded 
above is suitable only for the description of non coincident branes. In the case, often 
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considered in string theory, of stacks of overlapping branes, one would need a non Abelian 
generalization of the above construction which is not yet available [19]. 

In the brane Hitchin sigma model studied in this paper, branes are generic generalized 
complex submanifolds. So, we call them generalized branes. 

The action Sw of the model contains a topological Wess-Zumino term defined up to 
the relative periods of the closed relative 3-form {H,F). In the quantum path integral, 
so, the weight e^p(y/^Sw) is unambiguously defined only if {H/2tt,F/2'k) has integer 
relative periods. Therefore, the relative cohomology class [{H/2t:,F/2t:)\ belongs to the 
image of H^{M,W,'L) in H^{M,W,B). This indicates that generalized branes support 
relative gerbes with connection and curving [42]. 

In the A and B topological sigma models with branes, the H field vanishes and the 
branes support Abelian gauge fields of field strength F. The ends of open strings lie on the 
branes and carry Chan-Paton point charges, which couple to gauge fields of the branes. 

As is well known, the consistent interaction of a point charge with a gauge field requires 
that the gauge field strength has quantized fluxes. Since the open strings carry Chan-Paton 
point charges coupling to gauge flelds of the branes, the brane gauge curvature F/2ti has 
integer periods. It follows that the A- and B-branes always support line bundles with 
connections [19]. 

Since open strings are involved, the world sheets have non empty boundaries and 
the world sheet flelds obey boundary conditions, whose nature depends on the topolog- 
ical twisting and restricts the type of submanifolds of the Kaehler target space branes 
can be [21]. In the A model, in its simplest form, the boundary conditions are such that 
the complex structure exchanges normal and tangent directions to the brane and, for this 
reason, an A-brane must be a Lagrangian submanifold with respect to the Kaehler sym- 
plectic structure. In the B model, conversely, the boundary conditions make the complex 
structure preserve normal and tangent directions and, so, a B-brane must be a complex 
submanifold with respect to the Kaehler complex structure. 

Further restrictions arise from reqinrcmcnt that the coupling of the open string Chan- 
Paton charges to the brane gauge flelds is invariant under the nilpotent topological charge 
[19]. In A model, this implies that the fleld strength F vanishes, so that the underlying 
line bundle is fiat. In the B model, conversely, this makes F be a (1, 1) 2-form and the 
underlying line bundle holomorphic. 

Other restrictions follow from the requirement of ghost number anomaly cancellation. 
In the A model, this requires that the branes are graded Lagrangian submanifolds of 
vanishing Maslov class [20] . 
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To summarize, the above qualitative discussion indicates that an A-brane is a La- 
grangian submanifold carrying a flat hne bundle with flat connection and that a B-brane 
is a complex submanifold carrying a holomorphic line bundle with connection with (1,1) 
curvature. The careful analysis of Kapustin and Orlov [22] shows that, in the A model, a 
class of coisotropic submanifolds carrying non flat line bundles can also serve as A-branes, 
at least at the classical level. (See also the recent proposal of ref. [15].) 

The above brief review of how branes show up in the A and B topological sigma models 
fits quite well Gualtieri's formalism of generalized complex submanifolds illustrated above. 
Indeed, A- and B-branes are generalized complex submanifolds of the types described in 
the paragraphs of eqs. (3.9a) and (3.8a), respectively, and, thus, also particular examples 
of generalized branes 



4. 2— dimensional de Rham superfields 



In general, the fields of a 2-dimensional field theory are differential forms on a oriented 
2-dimensional manifold E. They can be viewed as elements of the space Fun(nTS) of 
functions on the parity reversed tangent bundle IITE of E, which we shall call de Rham 
superfields [31,35]. More explicitly, we associate with the coordinates of E Grassmann 
odd partners with 

deg^" = 0, degC" = 1. (4.1) 

transforms as the differential of 2" under coordinate changes. A de Rham superfield 

•0(;2,C) is a triplet formed by a 0-, 1-, 2-form field '0'^°''(2;), 'il)'^^^ (y_{z) ^ ^/;'^^^q,^(;2) set as 

^{zX) = ^^"^z) + C^^^\{z) + \CC^'^^^Kp{z). (4.2) 

The forms i/j^'^^ are called the de Rham components of ip. 

IITE is endowed with a natural differential d defined by 

dz"" = C, dC = 0. (4.3) 

In this way, the exterior differential d of E can be identified with the operator 

d = C^. (4.4) 

The coordinate invariant integration measure of IITE is 

li = dz^dz^dC^dC^. (4.5) 
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Any de Rham superfield ip can be integrated on IITE according to the prescription 

/ /iV= / |dz«d^^V^'^a/3(-s)- (4.6) 

Similarly, fields on the boundary dT, can be viewed as elements of the space Fun(nT 
d'E) of functions on the parity reversed tangent bundle UTdT, of dT,, which we shall 
call boundary de Rham superfields. Again, we associate with the coordinates s of dT, 
Grassmann odd partners <; with 

degs = 0, deg^^ ^ 1. (4.7) 

^ transforms as the differential of s under coordinate changes. A generic boundary de Rham 
superfield xi^i^) is a doublet formed by a 0-, 1-form field x^^K^)i X^^'^sis), organized as 

X{s,<;) = x^'\s)+,x^'\{s). (4.8) 

The forms x^'^K x''^'* called the boundary de Rham components of x- 
HTdT, is endowed with a natural differential dg defined by 

dds = ^, dd<; = 0. (4.9) 

The exterior differential dg of dT, can be identified in this way with the operator 

(4.10) 

The coordinate invariant integration measure of IlTdT, is 

fid = dsd<i. (4.11) 
Any de Rham superfield x can be integrated on IIT^E according to the prescription 

fXdX= <f dsx^'\{s). (4.12) 
Let ig : dT, — > S be the natural injection. If •0 is a de Rham superfield of E, then 

L9*^{s, <;) = V^°n^a(«)) + <^^{s)^^'K{id{s)) (4.13) 
is a boundary de Rham superfield, the pull-back of ip. By Stokes' theorem, 

Hdip = (b ^gig*-^. (4-14) 

nTE JuTdH 

17 



Often, for the sake of simplicity, we shall write V' rather than tg*ip in the right hand side. 
Similarly, by the boundary Stokes' theorem, if x is a boundary de Rham superfield, 

/ fidddX = 0. (4.15) 

It is possible to define functional derivatives of functionals of de Rham superfields. 
Let t/^ he a, de Rham superfield and let F{'i(;) be a functional of ip. We define the left/right 
functional derivative superfields 6i^rF{'4')/^'4' ^ follows. Let u be a superfield of the same 
properties as ip. Then, 

—F{ijj + ta) = i^a—-—= l^—F-i — ^- (4-16) 
at t=o Jyitt. Jutt. 

To write dF {ip-^ta) / dt\^_^ as an integral on IITE as above, repeated applications of Stokes' 
theorem are required. This yields boundary terms, which must cancel out. In general, this 
is possible only if suitable boundary conditions are imposed in the superfields. 

In the applications below, the components of the relevant de Rham superfields carry, 
besides the form degree, also a ghost degree. We shall limit ourselves to homogeneous 
superfields. A de Rham superfield is said homogeneous if the sum of the form and 
ghost degree is the same for all its components of ip. The common value 

of that sum is called the (total) degree degip of ip. It is easy to sec that the differential 
operator d and the integration operator Jyixj: A* carry degree 1 and —2, respectively. Also, 
if F('0) is a functional of a superfield •0, then degSi^rF{ilj)/dip = degF — degi/; + 2. Similar 
considerations hold for the boundary de Rham superfields. In this case, the differential 
operator dg and the integration operator J^^^^fig carry degree 1 and —1, respectively. 

5. The Hitchin sigma model in the presence of branes 

In this section, we shall first review the formulation of the Hitchin sigma model in the 
absence of branes [31]. Subsequently, we shall show how to incorporate branes into it. 

The Hitchin sigma model is closely related to the standard Poisson sigma model [33- 
34], of which it has the same field content. The approach used here is based on the 
Batalin-Vilkovisky quantization scheme [35-37]. To make the treatment as simple and 
transparent as possible, we shall use the convenient de Rham superfield formalism (cf. 
sect. 4) following the original work of Cattaneo and Felder in [35] (see also [45-47]). We 
shall limit ourselves to the lowest order in perturbation theory, since the constraints on 
target space geometry following from the Batalin-Vilkovisky classical master equation lead 
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directly to Hitchin's generalized complex geometry. Quantum corrections will presumably 
yield a deformation of the latter, whose study is beyond the scope of this paper. We will 
not attempt the gauge fixing of the field theory, which, at any rate, is expected to be 
essentially identical to that of the ordinary Poisson sigma model as described in [35,45]. 
We shall consider directly the twisted version of the Hitchin sigma model. The umtwisted 
version can be readily obtained by setting the H field to zero. 

When branes are absent, the target space of the Hitchin sigma model is a twisted 
manifold (M, H) (of. sect. 2). The base space is a closed oriented surface S. 

The basic fields of the standard Hitchin sigma model are a degree superembedding 
X G r(HTE, M) and a degree 1 supersection y G r(HTE, x*HT*M), where H is the parity 
inversion operator. With respect to each local coordinate of M, x, y are given as de 
Rham superfields x", ya- Under a change of coordinates, these transform as 



y'a = d'at''ot-\x)yb 



(5.1) 
(5.2) 



The resulting transformation rules of the de Rham components of x"'{z,Qi ya{z,C) ^re 
obtainable by expanding these relations in powers of C'*- 
The Batalin-Vilkovisky odd symplectic form is 



'llTS 



5x°-5ya + \Hahc{x)5x°'dx^5x' 



(5.3) 



n is not of the canonical form when H ^ 0. Hence, x", ya are not canonical fields/antifields. 
However, 1] is a closed functional form, SQ = 0. In this way, one can define antibrackets 
( , ) in standard fashion. The resulting expression is 



iF,G) 



5rF5iG 6rF6iG ^ ^ JrF ,6iG 

- Habc{x)- dx" 



5x"- 5ya Sya 5x°- 



Sya ^Vc 



(5.4) 



for any two functionals F, G oi x^, ya- 

In the Hitchin sigma model, the target space geometry is specified by a generalized 
almost complex structure J (cf.sect. 2). In the representation (2.11), the action is 



S= I II 
- 2 



yadx" + \P''\x)yayb + \Qab{x)dx''dx^ + J''b{x)yadx^ 
x^^'^H. 



(5.5) 
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Here, F is a 3-fold such that 



dT = S. (5.6) 

x^^^ : r ^ M is an embedding such that x^^^\y: equals the lowest degree component x^*^-* 
of the superembedding x. The last H dependent term is a Wess-Zumino like term. A 
similar term was added to the action of the standard Poisson sigma model in ref. [48] . Its 
value depends on the the choice of x^^\ In the quantum theory, in order to have a well 
defined weight exp(-\/^S') in the path integral, it is necessary to require that H/^tt has 
integer periods, so that the cohomology class [11/27:] e iy^(M, R) belongs to the image of 
H^{M,Z) in H^{M,R) (cf. sect. 2). We note that the above definition of the topological 
term works only if x^^\Tj) is a boundary in M. If one wants to extend the definition to 
the general case where a;(°)(S) is a cycle of M, the theory of Cheeger-Simons differential 
characters is required [49,50]. 

A straightforward computation furnishes 

iS,S) = 2 [ n\- \AH'''"'{x)yaybyc + \BHj"'{x)dx^ybyc (5.7) 

- \CHah'{x)dx''dx^yc+\DHabc{x)dx''dx^dx'' 

— \Habc{x)dx"'dx^dx'^ 

where the tensors Ah, Bh, Ch, Dh are given by (2.18a-(i). Hence, S satisfies the classical 
Batalin-Vilkovisky master equation 

(5, 5) = (5.8) 

if {2.na-d) hold, i.e. when J" is an twisted generalized complex structure and, so, 
(M, J) a twisted generalized complex manifold. (Recall that dx°'dx^dx^ = on HTS.) 
This shows that there is a non trivial connection between generalized complex geometry 
and quantization a la Batalin-Vilkovisky of the sigma model [31]. {2.17 a-d) are sufficient 
but not necessary conditions for the fulfillment of the master equation (5.8). 
The Batalin-Vilkovisky variations Sbv^"', ^Bvya are given by 

Sbvx'' = iS,x''), (5.9a) 
SBvya= {S,ya)- (5.96) 
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Using (5.4), we can then derive the expressions of the Batahn-Vilkovisky variations 6bvx°', 
^BvVa- The result is 



Sbvx" = dx" + P°'\x)yb + J''b{x)dx\ (5.10a) 

SsWa = dya + \daP^^{x)ybyc + \{daQhc + dhQca + dcQab){x)dx''dx^ (5.106) 
+ {daJ\ - dcJ\){x)ybdx'' + J\{x)dyk 

+ \{HabdJ''c - HacdJ\){x)dx^dx'' + H adcP''\x)ybdx^ ■ 

As well-known [36,37], the Batalin-Vilkovisky variation operator 5bv is nilpotent, 

5bv'^ = 0. (5.11) 

The associated cohomology is the classical Batalin-Vilkovisky cohomology. Also, by (5.8) 

5bvS = 0. (5.12) 

It is interesting to see how the odd symplectic form O behaves under a h transform of 
the H field of the form (2.13). It turns out that a meaningful comparison of the resulting 
symplectic form O' and the original symplectic form requires that the superfields x", ya 
also must undergo a h transform of the form 

x"" = x", (5.13a) 

y'a =ya + bab{x)dx''. (5.136) 

Q.' = n. (5.14) 



It is then simple to verify that 



If the 2-form b is closed, then H' = H, by (2.4). In that case, the b transform is canonical, 
i. e. it leaves the Batalin-Vilkovisky odd symplectic form (5.3) invariant. 

It is similarly interesting to see how the action S behaves under a b transform of the 
H field and of the generalized almost complex structure J of the form (2.13), (2.16a-c). 
Provided the field redefinition (5.13a, b) are carried out, one finds that the resulting action 
S' action equals the original one S, 

S' = S. (5.15) 
21 



So, b transform is a duality symmetry of the Hitchin sigma model [31]. ^ 

The Batalin-Vilkovisky variation operator Sbv behaves covariantly under b transfor- 
mation in the following sense. Let H' and J' he a b transform of the H field and of the 
generalized complex structure J given in (2.13), (2.16a-c). Let x'°', y'a be the b transform 
of the superfields x"', ya given in (5.13a, b). Let S'^yx'"', S'^yy'a be given by (5.10a, b) with 
x"', ya and Habc, J^'b Qab replaced by x'"', y'a H'abc, J'^'b Q'ab, respectively. 

Then, one has the relations 

S'bvx'" = Sbvx'', (5.16a) 

S'svy'a = SsviVa + bab{x)dx^) . (5.166) 

(Compare (5.16a, b) with (5.13a, b).) In this sense, b transformation is compatible with 
the Batalin-Vilkovisky Sbv cohomological structure. 

Let see next how one can incorporate branes into the Hitchin sigma model. The 
discussion at the end of sect. 3 shows that, if we want to make contact with the usual 
picture of branes, a brane contained in the target bulk space should not be seen simply as 
an ordinary submanifold W of the manifold M, but rather as a generalized submanifold 
(W, F) of the twisted manifold (M, H). 

A sigma model with branes describes the dynamics of open strings whose ends lie on 
the branes. Therefore, the world sheet E of our sigma model with branes must have a non 
empty boundary dT, and the superfields x, y must satisfy appropriate boundary conditions. 
These are properly expressed in terms of the pull-back superfields id*x, Lg*y (cf. sect. 4). 
Below, for notational simplicity, tg* will often be tacitly understood. 

The superembedding x must be such that Lg*x{IlTd'E) C W. As a consequence, the 
superfield x°' must satisfy the boundary condition 

xP = (5.17a) 

along IlTdT,, in adapted coordinates (cf. sect. 3). From a geometrical point of view, it 
is also natural to demand that the pull-back superfields Lg*dx, Lg*y represent a section of 



^ In the untwisted Hitchin model, the H field is absent and the action S has no topo- 
logical term. In that case the 2-form b must be closed and S' = S — 2 J^x^^^*b, i. e. 
S, S' differ by a topological term. If 6/27r has integer periods and, so, describes a gerbe 
gauge transformation, one has exp{^/^S') = exp{^/—lS) in the quantum path integral. 
b transform is then a duality symmetry of the quantum Hitchin sigma model. See [31]. 
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the pull-back {Lg*x)*IlT^W of the parity reversed generalized tangent bundle IIT^W of 
W (cf. sect. 3). On account of (3.6a, 6), one has the further boundary condition 



Vi + Fij{x)ddX^ = 



(5.176) 



along UTdT,, in adapted coordinates. (The condition dgx^ = is already implied by 
(5.17a).) The boundary conditions (5.17a, b) are purely geometrical hence kinematical. 

When M contains a brane W, the expression (5.3) of the Batalin-Vilkovisky odd 
symplectic fl is no longer correct. As it stands, fl fails to be closed, as in fact 



5VL = - (b Hd \diFjjf{x)8x''8x^ 5x^ . 



(5.18) 



The right hand side is a boundary term and can be compensated for by modifying (5.3) 
by a boundary term. The resulting expression is 



w 



Sx^Spa + \Habc{x)5x"'dx^5x'' 



+ f iJiQ\Fij{x)5x'5x^ . (5.19) 



Now, Vt\Y is a closed functional form, dVtw = 0. In this way, one can define antibrackets 
{■,)w in standard fashion. The resulting expression is 



w 



SrFSlG SrFSlG r..JrF , j,5lG 



Sx"- 5ya Sya Sx" 
5rF SiG 



Habc{x)^ dx 



Sya Syc 



(5.20) 



JuTdT, dyi dyj 



for any two functionals F, G of a;", ya- 

Now, let us introduce the Hitchin sigma model in the presence of a brane (W^F). 
The target space geometry of the bulk (M, H) is specified again by a generalized almost 
complex structure J'. {W, F) is assumed to be a generalized almost complex submanifold 
of {M,H,J') (cf. sect. 3). This assumption is natural from a geometrical point of view. 
In the representation (2.11), the action of the model reads 

Sw= [ ^lyadx"" + ^P''\x)yayb + lQab{x)dx''dx^ + J\{x)yadx''] (5.21) 
-2 J x^^^*H + 2 J x^^^F. 

Here, F, A are respectively a 3- and a 2-fold such that A C ^F, 
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ar - A = s, aA = -as (5.22) 

^(0) : r — > M is an embedding such that x(°)(A) C W and that equals the lowest 

degree component x^^^ of the superembedding x. The last two terms dependent on H, 
F constitute altogether a Wess-Zumino like term. Their total value depends on the choice 
of x^^\ In the quantum theory, in order to have a well defined weight exp{^/^Sw) in the 
path integral, it is necessary to require that {H/27r, F/27r) has integer relative periods, so 
that the relative cohomology class [(ii/"/27r, F/27r)] belongs to the image of H^{M,W,Z) 
in iy^(M, W,]R) (cf. sect. 3). The above definition of the topological term works only if 
(x^^\Tj) , x^^^dT,)) is a relative boundary in M. If one wants to extend the definition to 
the general case where (x^^\T,), x^^\dT,)) is a relative cycle of M, the theory of Cheeger- 
Simons relative differential characters is required [51]. 

The brane action Sw given in (5.21) is the most obvious generalization of the no brane 
action S given in (5.5). It is so designed to yield the same expression for the derivatives 
^'^Sx'^ , ^''sy^ of Sw and the derivatives %^ of S upon imposing a natural boundary 

condition on the fields as we show next. Using the general definition (4.16) to compute 
^^dx"^ , one obtains an integral on IITS and a boundary integral on IlTdTi of the form 



B 



.I. = -i> ii9Fij{x)[ddX^ +J^k{x)d9x'' + P^P{x)yp-P^''Fki{x)d9x']Sx\ (5.23) 



in adapted coordinates. The calculation involves repeated applications of the relations 
{3.7 a-c) which follow from (VF, F) being a generalized almost complex submanifold of 
{M,H,J') and the kinematical boundary conditions (5.17a, b). Prom (5.23), it seems 
reasonable to demand that 

Fij{x) [dgx^ + J\{x)ddx'' + P'^{x)yp - P^''Fki{x)d9x'] = (5.24) 

along UTd'E. ^ Unfortunately, this boundary condition suffers a number of diseases to be 
discussed below. These are cured by replacing (5.24) by a stronger boundary condition 
implying (5.24), namely 

dax' + r,{x)ddx' + P''{x)yp - P'^ Fjk{x)dQX^ = (5.25) 



^ Of course, this condition is no longer necessary when = 0. We shall not consider 
this possibility in the following discussion. 



24 



along nras. Using again the general definition (4.16) to compute '■J' ^ , one obtains 
again an integral on IITS and no boundary integral on IlTdT,. Therefore, no further 
boundary conditions are required. The boundary conditions (5.24), (5.25) originate from 
the structure of the brane action Sw and are therefore dynamical. 

The expressions of ^^gy^ obtained in this way are given respectively by the 

right hand side of (5.106) and plus/minus the the right hand side of (5.10a). Using these, 
one can compute the antibrackets {Sw, Sw)w using (5.20). The calculation involves re- 
peated applications of the relations (3.7a-c) and the kinematical boundary conditions 
(5.17a, b) again, and the dynamical boundary condition (5.24) or (5.25). (One also uses 
that dgx^dgx^ = on the 1-dimensional manifold dT,.) Because of (5.24), the boundary 
term in the right hand side of (5.20) vanishes identically. The result is 

(Sw, Sw)w = 2 / 1^1 - lAH''''%x)yaybyc + \BHj'%x)dx''yi,yc (5.26) 

- \CHab'{x)dx''dx^yc + lDHahc{x)dx''dx^dx'' 

- \Hahc{x)dx"-dx^dx'' 

where the tensors Ah, Bh, Ch, Dh are given by (2.18a-(i). Note that the expression (5.26) 
of the brackets {Sw, Sw)w is formally identical to the expression (5.7) of the brackets 
{S,S). Hence, Sw satisfies the brane classical Batalin-Vilkovisky master equation 

{Sw, Sw)w = 0, (5.27) 

if (2.17a-d) hold, i.e. when J is an H twisted generalized complex structure. In this way, 
the brane (W, F) is a generalized complex submanifold of the twisted generalized complex 
manifold {M,H,J) (cf. sect. 3). This shows that the non trivial connection between 
generalized complex geometry and quantization a la Batalin-Vilkovisky of the sigma model 
continues to hold also when M contains a brane W . This is so by design. 
The Batalin-Vilkovisky variations 5bvwx°', SsYwya are given by 

Sbvwx'' = {Sw,x"-)w, (5.28a) 
SBVwVa = {Sw,ya)w- (5.286) 



^ Exploiting (5.17a, b), it is straightforward to show that both conditions (5.24), (5.25) 
are adapted covariant. 
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Using (5.20), we can then derive the expressions of the brane Batalin-Vilkovisky variations 
5bvwx°'i SBVwUa- They are given by the same formulae as those of the no brane Batahn- 
Vilkovisky variations 5bvx°', Ssvya given in (5.10a, b). Again, this is so by design. As 
Sbv (cf- eq. (5.11)), the brane Batahn-Vilkovisky variation operator Sbvw is nilpotent 

Sbvw^ = 0. (5.29) 

The associated cohomofogy is the brane classical Batalin-Vilkovisky cohomology. This 
relation is not the trivial restatement of (5.11), which it may appear at first glance, since, 
in the presence of branes, the superfields x"", ya obey the boundary conditions derived 
above. (The natural question about the compatibility of the boundary conditions with the 
Batalin-Vilkovisky cohomological structure will be discussed in the next section.) Also, 
one has 

SbvwSw = 0, (5.30) 

by (5.27). 

It is interesting to check whether the incorporation of branes into the Hitchin sigma 
model is compatible with the b transformation symmetry. 

To begin with, let us see how the odd symplectic form behaves under a b transform 
of the H and F fields of the form (2.13), (3.5). Again, a meaningful comparison of the 
resulting symplectic form Q\y and the original symplectic form Qw requires that the 
superfields x°', ya undergo the b transform (5.13a, b). From (5.19), it is straightforward to 
verify that 

Q\y = Qw, (5.31) 

generalizing (5.14). 

Next, let us see how the action Sw behaves under a b transform of the H and F fields 
and of the generalized almost complex structure J of the form (2.13), (3.5), (2.16a-c). 
Using (5.13a, 6), one finds that 

S{y = Sw, (5.32) 

generalizing (5.15). So, b transform is a duality symmetry of the Hitchin sigma model even 
in the presence of a brane. 
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Expectedly, the brane Batalin-Vilkovisky variation operator Sbvw behaves covari- 
antly under b transformation as in the no brane case. Relations (5.16a, b) hold true with 
^bvj ^'bv replaced by SbvWj ^'bvwi respectively, where S'^yy^ is defined analogously 
to Thus, b transformation is compatible with the Batalin-Vilkovisky cohomological 

structure also in the brane case. 

In the presence of a brane W, the superfields a;", pa obey the boundary conditions 
(5.17a, b) and (5.24) or (5.25). It is important to check whether these conditions are 
invariant under b transformation. 

Let us consider first the kinematical boundary conditions (5.17a, b). Requiring their b 
invariance amounts to demanding that whenever the x", i/a satisfy (5.17a, b) with respect 
to the brane field F, their b transforms x'", y'a, given by (5.13a, 6), satisfy (5.17a, b) with 
respect to the b transformed brane field F' , given by (3.5). It is trivial to check that this 
is indeed the case. 

Let us consider next the dynamical boundary conditions (5.24) or (5.25). Again, 
requiring their b invariance amounts to demanding that whenever the x", ya satisfy (5.24) 
or (5.25) with respect to the fields H and F and the generalized almost complex structure 
J', their b transforms x'", y'a, given by (5.13a, 6), satisfy (5.24) or (5.25) with respect 
to the b transformed fields H' and F' and the b transformed generalized almost complex 
structure J'', given by (2.13), (3.5), (2.16a-c). It is trivial to check that this is the case 
for the boundary condition (5.25) but not for the boundary condition (5.24). The latter 
therefore breaks b invariance. b transformation symmetry is one of the most basic features 
of both generalized complex geometry and the Hitchin sigma model. Renouncing to it 
seems to be out of question. This indicates that (5.25) is the appropriate dynamical 
boundary condition. 

6. The classical BV cohomology in the presence of branes 

One of the most interesting aspects of the Hitchin sigma model is the associated 
classical Batalin-Vilkovisky cohomology, which describes the observables of the model at 
semiclassical level [36,37]. As we have seen above, when a brane W is present, the expres- 
sions of the brane Batalin-Vilkovisky variations Sbvwx"", ^BVwVa are the same as those 
of the no brane variations Sbv^""-, ^BvVa given in (5.10a, b). One would thus conclude that 
the no brane and brane classical Batalin-Vilkovisky cohomology are identical. However, 
one should keep in mind that, in the brane case the basis fields a;'*, ya of the model obey the 
kinematical boundary conditions (5.17a, b) and the dynamical boundary conditions (5.24) 
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or (5.25). It is therefore important to check the compatibihty of the boundary conditions 
and the overall Batalin-Vilkovisky cohomological structure. 

Concretely, this is done as follows. The boundary conditions have the general form 

3'{td*x,td*y) = 0, (6.1) 

where 3^ is some functional of the boundary superfields. The boundary conditions are 
compatible with Batalin-Vilkovisky cohomological structure provided 

6Bvw3'{Ld*x,Ld*y) = 0, (6.2) 

when (6.1) holds, where the left hand side is computed using (5.10a, b) and the fact that 
SBVw^-d* = Ld*SBVW- If (6.2) does not hold, then (6.2) becomes a new set of boundary 
conditions to be added to the original ones. The compatibility check then must be carried 
out again from the beginning with the enlarged set of boundary conditions so obtained. 
The process must be continued until (6.2) is satisfied identically. Alternatively, one may 
replace the boundary conditions (6.1) with stronger boundary conditions implying (6.1), 
for which (6.2) holds identically. (These will be expressed in terms of some functional 
different from 3".) 

We consider first the kinematical boundary conditions (5.17a, b). Their compatibility 
with the Batalin-Vilkovisky cohomological structure requires that 

Sbvwx'' — 0, (6.3a) 
^BVwVi - Fij{x)ddSBvwx^ + dkFij{x)5Bvwx^ddX^ = (6.36) 

along IlTdT,, in adapted coordinates, when (5.17a, b) hold. Using (3.7a-c) and (5.17a, b) 
it is straightforward to verify that these relations do indeed hold true. 

We consider next the dynamical boundary conditions (5.24) or (5.25). In the previous 
section, we saw that the basic requirement of b transformation symmetry invariance of the 
dynamical boundary conditions rules out (5.24) and selects (5.25) as the only consistent 
condition. We now shall show that the requirement of compatibility of the dynamical 
boundary conditions with the Batalin-Vilkovisky cohomological structure leads to the 
very same conclusion. 

By (5.10a), the condition (5.24) can be written concisely as 

Fij(x)SBvwx' = (6.4) 
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along UTdT,. Proceeding as above, its compatibility with the Batalin-Vilkovisky cohomo- 
logical structure requires that 

dkFij{x)5Bvwx^5Bvwx^ = (6.5) 

along UTdT,, when (5.17a, b) and (5.24) hold. Explicitly written, (6.5) reads 

{dkFij + djFki)P'"'{x)y, [dox^ + Jh{x)dax' + \P'P{x)yp - P^' Fira{x)ddX^] = (6.6) 

along IlTd'E. (6.6) is not satisfied identically, when (5.17a, b) and (5.24) hold. According 
to the procedure described above, (6.6) should be added as a new boundary condition. 
Prom (5.29), (6.5), it is easy to see that no further boundary conditions would be required 
by compatibility. The alternative condition (5.25) can be written concisely 

Sbvwx-' = (6.7) 

along UTdT. By (5.29), (6.7), the boundary condition (5.25) satisfies the compatibility 
requirements (6.2) trivially. 

In conclusion, (5.24) involves the introduction of the new dynamical boundary con- 
dition (6.6). (6.6) is rather complicated and does not seem to have any obvious natural 
interpretation. (5.25), conversely, does not have this undesirable feature. This analysis 
provides conclusive evidence that (5.25), not (5.24), is the appropriate consistent dynam- 
ical boundary condition of Hitchin sigma model. Combining (6.3a), (6.7) one gets the 
boundary condition 

Sbvwx" = (6.8) 

along UTdT, where the left hand side is given by (5.10a). This seems to be a more 
transparent set of boundary conditions, also in the light of the fact that the y classical 
field equations of the Hitchin sigma model can be written precisely as Sbvwx'^ = on 
UTT. 

In ref. [31], the classical Batalin-Vilkovisky cohomology of the no brane Hitchin sigma 
model was studied. In particular, it was shown that there exists a natural homomorphism 
of the generalized Dolbeault cohomology of target space into the Batalin-Vilkovisky co- 
homology depending on a choice of a singular supercycle Z of the world sheet E. The 
analysis was based on an adaptation of the well-known descent formalism of gauge theory. 
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The homomorphism was constructed by associating with a representative of a general- 
ized Dolbeault cohomology class a local superfield and integrating the latter on Z and by 
showing that the results was a representative of a Batalin-Vilkovisky cohomology class. 
It was also found that the generalized Dolbeault cohomology contains the cohomology of 
the generalized deformation complex as a subcohomology. We shall not repeat the details 
of this analysis. The interested reader is invited to read sect. 6 of ref. [31] for a thorough 
discussion of these matters. 

One may wonder whether and how the presence of branes modifies the classical 
Batalin-Vilkovisky cohomology. One may think, for instance, that a relative version of 
the generalized Dolbeault cohomology may be the appropriate cohomology in this case, 
since relative cohomology involves typically a space and a subspace, here the target bulk 
and the brane. This is unlikely to work out for the following simple reason. A generalized 
complex submanifold of a twisted generalized complex manifold is not itself a twisted gen- 
eralized complex manifold in general, while any reasonable relative version of a cohomology 
requires that the two spaces involved carry the same type of structure. 

A more likely scenario is the following. In the presence of branes, the Batalin- 
Vilkovisky cohomology will contain a sector reproducing the no brane cohomology as- 
sociated with the generalized Dolbeault cohomology of the bulk and a further sector based 
on some cohomological structure of the brane, whose nature is to be determined. We 
devote the rest of this section to the analysis of this matter. The discussion parallels that 
of the no brane cohomology of ref. [31] with some significant differences to be discussed 
below. 

The construction expounded in the following involves the singular chain complex of 
9E. Since we deal with boundary de Rham superfields throughout (cf. sect. 4), it is 
convenient to use the boundary singular superchains formalism. A boundary singular 
superchain C is a doublet formed by a 0-, 1-dimensional singular chain C'(o), C(i) of dH 
organized as a formal chain sum C = C(o)-|-C'(i). The nilpotent singular boundary operator 
d of extends to boundary superchains in obvious fashion by setting (5aC)(o) = dC(^i^, 
{ddC)(^i) = 0. A boundary singular supercycle Z is a superchain such that dgZ = 0. A 
boundary de Rham superfield x can be integrated on a boundary superchain C: 

I X = I / dsx^^\{s). (6.9) 

Stokes' theorem holds, dgx = Jg^c particular, one has dgx = for any 

boundary supercycle Z. 
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We call a boundary de Rham superfield X local, if it is a local functional of the pull- 
back of the basic superfields Ld*x, id*y. Let X be some local boundary superfield and let 
there be another local boundary superfield Y such that 



5bvwX = dgY. (6.10) 

Thus, X defines a mod dg Batalin-Vilkovisky cohomology class. Then, if Z is a boundary 
singular supercycle, one has 



Sbvw / X= / daF = 0, (6.11) 
Jz J z 

by Stokes' theorem. It follows that 

(Z,X)a= / X (6.12) 
Jz 

defines a Batalin-Vilkovisky cohomology class. A standard analysis shows that this class 
depends only on the mod dg Batalin-Vilkovisky cohomology class of X. So, one may 
obtain Batalin-Vilkovisky cohomology classes by constructing boundary local superfields 
X satisfying (6.10). 
Define 

dd^\ [dd - ^/^{Sbvw - dg)] (6.13) 

and its complex conjugate dg, where here and below the operator 5bvw is tacitly restricted 
to the space of local boundary superfields. Prom (5.29), using that dg5 bvw + ^ bvw dg — 0, 
it is immediate to check that 

dg^ = 0, (6.14a) 
dg'^ = 0, (6.146) 

dgdg + dgdg^O. (6.14c) 

From (6.5), one has further 

dg = dg + dg, (6.15a) 

Sbvw = dg + dg + V^{dg - dg). (6.156) 
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The operator dg acts on the space of local boundary superfields, carries degree 1, by 
(6.13), and it squares to 0, by (6.146). Therefore, one can define a dg local boundary 
superfield cohomology in obvious fashion. 

Let X be a local boundary superfield such that 

dgX = 0. (6.16) 

X defines a dg local boundary superfield cohomology class. By (6.15a, 6), X satisfies (6.10) 
with Y = (1 + \^—l)X. So, as shown above, for any boundary supercycle Z, {Z,X)g 
defines a Batalin-Vilkovisky cohomology class. If X = dgU for some local boundary 
superfield C/, so that the corresponding dg cohomology class is trivial, then {Z,X)g ~ 
\^/^-i5BVw{Z,U)g, by (6.13) and Stokes' theorem and, so, the corresponding Batalin- 
Vilkovisky class is trivial too. Therefore, for any boundary singular supercycle Z, there 
is a well-defined homomorphism from the dg local boundary superfield cohomology into 
the Batalin-Vilkovisky cohomology. This homomorphism depends only on the boundary 
singular homology class of Z. 

Adapting the analysis of ref. [31], we carry out the construction of the brane sector of 
the Batalin-Vilkovisky cohomology by associating with a given field of the brane a local 
boundary superfield and imposing that the latter satisfies (6.16). Taking into account the 
boundary conditions (5.17a, 6), we see that the basic boundary superfields at our disposal 
are the x*, Hp, in adapted coordinates. (Here and below, the pull-back operator ig* is 
tacitly understood.) Tentatively, we may consider local boundary superfields of the form 

p,g>o 

where T e ®p^q>QC°^{N^NW ®^'iT''W ®C), NW = TM\w/TW being the normal bundle 
of W. This is however too naive. In fact, under a change of adapted coordinates (cf. sect. 
3), the dgx'^ transform into themselves, since dgx'^ = by (5.17a), while the tjp do not 
but mix with the Fij{x)dgx^ , since yi = —Fij{x)dgx^ by (5.176). For this reason, an 
object of the form (6.17) would not be adapted covariant (cf. sect. 3). To get adapted 
covariant expressions, the components T^^ ^'^Pi^. of the brane field T must have more 
exotic transformation properties than those simplemindedly implied above. Formally, these 
are given as follows. T G ^wo* — ®n>o^wo^ ■, where ^wo^ is the space of the formal 
adapted covariant objects 

E iT^'"-'^n....^pi---^p.r •••r^ (6.18) 

p,q>0,p+q=n 
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rjp being anticommuting objects transforming as 



(6.19a) 
(6.196) 



under a change of adapted coordinates. The resulting transformation properties of the 
component fields T^i- -^Pjj...j^ are not tensorial and in explicit form are rather messy. 

We note that all terms in the right hand side of (6.17) with g > 1 vanish, since 
ddX^dgx^ = on the 1-dimensional manifold 9E. This property will be used repeatedly 
to suitably reshape the expressions derived below. 

A simple calculation using (5.10a, b), {3.7 a-c), (5.17a, b) yields 



Sbvwx' = ddx' + P"'{x)yp + {r, - P'''Fkj){x)d9X^, 
SbvwVp = ddVp + \dpP'^''{x)yayr + {dpJ'^i - diJ" p 

+ P"'Hijp - FijdpP^''){x)y^dx' + J%{x)ddya- 

From these relations and (6.13), (6.17), one finds 

ddXr= J2 ^dwr''' -'\,M^)ypi---ypJdx''---ddx'^ 

P,q>o 

+ Yl i^Kw''^''---'-ir...i,{x)ddyayp, - ■ ■Vp.dax'' ■ ■ -ddx'^ 

P,q>0 



where 



ll...ln 



i{(-l)^[%T^-^-,...,,] 

+ K-l)"5(? - l)x/^M[,^^-,,T''-''^ 1,1,3..,^ 
+ (-l)^'p5v^M[,J^i|,|Tl-l^-^-l 



l2■■■^q 



p[Pl\j\Q.^P2---Pp\. 

|(p-l)a,p[^i^^Tl-l^3...p.].^ 



(6.20a) 
(6.206) 



(6.21) 



(6.22a) 
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with 

Mi^j = -di{j''j - P'^'Fij) + dj{j\ - P^'Fu), (6.226) 

Mi'^p = dirp - dpJ\ - P^'H.jp + FijdpP^% (6.22c) 

Zijp = diQjp + djQpi + dpQij + di {FjkJ^p) - dj {FikJ^p) (6.22d) 

+ FikdpJ^j — FjkdpJ'^i + FikFjidpP'^^ 

— {J^i — P'^^ Fii^Hjkp + {J^ j — P'^^ Fij^Hikp, 

the brackets [• • •] denoting full antisymmetrization of all enclosed indices except for those 
between bars | • • • | , and 

K^-T^i...p..^ = i{(5% + ^/^J%)T-^-^-,...,, (6.22e) 
So, Xr satisfies (6.16), if 

^wT^'■■■'^^,...^,=0, (6.23a) 

K^cryP^-Pr.^ = 0. (6.236) 

It can be verified that eqs. (6.23a, b) are jointly adapted covariant. Note that they are so 
only together, since in fact eq. (6.23a) is not by itself adapted covariant if eq. (6.236) is 
not taken into account. 

It is straightforward to show that, for fixed p, eq. (6.22e) defines a linear operator 
Kw'' : ^wo^ ^wo^~^i for n > 0. Let be the intersection of the kernels of the 

operators Kw^ with p = 1, . . . ,d — dimW, for n > 0. Let further = ®n>o^w'^- 

Eq. (6.236) then states that T e ^w*- 

Using (6.22a-e), by a very lengthy algebraic verification exploiting systematically 
{2.17a-d) and (3.7a-c), one finds that eqs. {6.22a-d) define a linear operator dw '■ S^w"^ — ^ 
^TvK^+i and that 



dw'^ = on S^w*- 
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(6.24) 



Thus, the pair {^w*, dw) is a cochain complex, with which there is associated a cohomol- 
ogy H* {3t^w* 1 dw) 1 which we call the generalized complex submanifold cohomology of W. 
Eq. (6.23a) then states that T e ^w* is a representative of a class of this cohomology. 

It is easy to see that eq. (6.17) defines a homomorphism of the generalized complex 
submanifold cohomology of the brane W into the dg local boundary superfield cohomol- 
ogy. Recall that the latter is embedded in a sector of the classical Batalin-Vilkovisky 
cohomology associated with the presence of branes. Therefore, the brane sector of the 
classical Batalin-Vilkovisky cohomology is related non trivially to the generalized complex 
submanifold cohomology ofW. 

^w* is actually a graded algebra and dw is a derivation on this algebra. As a conse- 
quence, H*{^w* , dw) has a canonical ring structure. The cohomology ring H*{^w* , dw) 
characterizes the generalized complex submanifold W . To the best of our knowledge, it 
was hitherto unknown. 

The investigation of the intrinsic content of dw)-: obscured by the adapted 

coordinate expressions in terms of which we defined it, is beyond the scope of this paper 
and should better be left to the mathematicians. The remarkable connections emerging 
here between the brane sector of the Batalin-Vilkovisky cohomology of the Hitchin sigma 
model on one hand and various aspects of Hitchin's generalized complex geometry and 
Gualtieri's notion of generalized complex submanifold on the other should however be 
emphasized for its field theoretic and physical mathematical interest. 

7. Discussion 

The Alexandrov-Kontsevich-Schwartz-Zaboronsky formalism of ref. [52] is a method 
of constructing solutions of the Batalin-Vilkovisky classical master equation directly, with- 
out starting from a classical action with a set of symmetries, as is done in the Batalin- 
Vilkovisky framework. In ref. [38], using such formalism, Cattaneo and Felder managed 
to obtain the Batalin-Vilkovisky action of the Poisson sigma model. In ref. [39] , the same 
authors extended their analysis by including branes in the target Poisson manifold and 
showed that branes had to be coisotropic submanifolds. (See also refs. [53,54] for a related 
study.) In spirit, their approach is very close to the one of the present paper. This calls 
for a comparison of their results and those of we have obtained. 

We consider first the case where branes are absent and discuss the incorporation 
of branes later. Following [38], we view the standard Poisson sigma model as a field 
theory whose base space, target space and field configuration space are respectively a two 
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dimensional surface S, a Poisson manifold M with Poisson bivector P and the space of 
bundle maps = {x, y) : HTS UT*M. 

The supermanifold IIT*M has a canonical odd symplectic structure, or P-structure, 
defined by the canonical odd symplectic form uj = duadt"'. With a;, there are associated 
canonical odd Poisson brackets {,)^ in standard fashion. The algebra of functions on 
TiT*M with the odd brackets ( , is isomorphic to the algebra of multivector fields on M 
with the standard Schoutens-Nijenhius brackets. 

The space of field configurations inherits an odd symplectic structure from that of 
nT*M and, so, it also carries a P-structure. The associated odd symplectic form is 
obtained from uj by integration over IITE with respect to the usual supermeasure ^ (cf. 
eq. (4.5)) and is given by the first term in the right hand side of (5.3). With O, there are 
associated odd Poisson brackets ( , )n over the algebra of functions on field configuration 
space, called Batalin-Vilkovisky antibrackets in the physical literature. 

The Poisson bivector field P of M can be identified with a function on IIT*M satisfying 
(P, P)i^ = 0. Its Hamiltonian vector field is an odd vector field Qp such that Qp^ = and 
defines a so called Q-structure on IIT*M. 

The Poisson bivector P yields a function S' on field configuration space, again by 
integration over IITE with respect to satisfying (S", S')n = 0. S' is given by the second 
term in the right hand side of (5.5). Its Hamiltonian vector field is an odd vector field Qs' 
such that Qs'^ = 0, and, so, it defines a Q-structure on field configuration space. 

The base space IITE carries also a canonical Q-structure d corresponding to the usual 
de Rham differential on E. d induces a Q-structure d on field space in obvious fashion, d 
is Hamiltonian, as indeed d = Qsoi where Sq is the function on field configuration space 
defined by the first term in the right hand side of (5.5). 5*0 satisfies (5*0, So)n = 0. 

One verifies that (5", 5'o)n = 0. The sum 5" = 5*0 + 5" thus satisfies (5", 5")^ = 0. 
S is nothing but the Batalin-Vilkovisky action of the Poisson sigma model satisfying 
the Batalin-Vilkovisky master equation. Its Hamiltonian vector field Sbv = Qs is the 
Batalin-Vilkovisky variation operator. 

In the untwisted case H — 0, the construction of the Hitchin sigma model of ref. [31] 
closely parallels that of the Poisson sigma model outlined above. The field space is the 
same as that of the Poisson sigma model. The field space odd symplectic form is also the 
same, being derived from the same odd symplectic form oj of HT*M exactly in the same 
way. The relevant difi^erence is that the role of the Poisson bivector P is played here by the 
generalized complex structure J^, which does not correspond to any multivector in general. 
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but which still corresponds to some structure of target space. The overall framework is 
otherwise totally analogous. 

In the twisted case H ^ 0, the situations is more complicated. li H = duB is 
exact, we can b transform ilf to by setting b = B (cf. eq. (2.4)), recovering in this 
way the untwisted case, and follow the lines outlined above. If H is not exact, this is no 
longer possible. The symplectic form fi, given by (5.5) cannot be straightforwardly ob- 
tained from a corresponding structure of IIT*M. Further, the action functionals contains 
a topological Wess-Zumino term and so it involves an extra geometrical datum, the ex- 
tension of the embedding field a; to a 3-dimensional manifold F bounded by S. The spirit 
of the Alexandrov-Kontsevich-Schwartz-Zaboronsky formalism, however, is still fully dis- 
cernible, in spite of the technical differences. 

Let us now examine the case when branes are present. In ref. [39], the Poisson 
sigma model in the presence of branes is studied. It is found that branes are coisotropic 
submanifolds W of the target Poisson manifold (Af, P). For a given brane VF, the bound- 
ary conditions impose that the world sheet field (p maps UTdTi into the parity reversed 
conormal bundle IIN*W. Writting (/) = [x^y] as usual and using adapted coordinates, the 
conditions are given by (5.17a, h) with Fij = 0. 

In this work, we study similarly the Hitchin sigma model in the presence of branes. 
We find that branes are generalized complex submanifolds {W, F) of the target twisted 
generalized complex manifold {M,H,J') (cf. sect. 3). For a given {W,F), the boundary 
conditions impose that the world sheet fields (p maps IITcJE into the the parity reversed 
generalized tangent bundle IIT^W of W (cf. sects. 3, 5). Writting = {x,y) and using 
adapted coordinates again, the boundary conditions are given by (5.17a, b) and (5.25). 

Now, we recall that not all Poisson manifolds (M, P) are particular cases of twisted 
generalized complex manifolds {M,H,J'). Only those with pointwise invertible P are. 
For these, H = and is of the form (2.22) with Q = —P~^. For this reason, the 
Hitchin sigma model is only a partial generalization of the Poisson sigma model since 
it can reproduce the latter only in the case where the target space Poisson bivector is 
pointwise invertible [31]. 

Similarly, when (M, P) is a Poisson manifold with pointwise invertible P, not all 
coisotropic submanifolds W oi M are generalized complex submanifolds (W, F) for some 
F of the corresponding generalized complex manifold (M, H = O^J) with J of the form 
indicateded above. If we require, as it is reasonable to do, that F = 0, then W must be 
a Lagrangian submanifold of M in order to be generalized complex submanifold, a more 
restrictive condition than being a coisotropic submanifold. This follows easily from (3.9a, 
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c), by inspection. So, again, the brane Hitchin sigma model is only a partial generalization 
of the brane Poisson sigma model. 

As to the boundary conditions, the conditions (5.17a, b) with = reproduce those 
of ref. [39] as already noticed. We have, however, the extra condition (5.25) with J^j = 
and Fij = 0. This condition follows from the requirement that the boundary integral (5.23) 
vanishes. Of course, when F = 0, this condition is no longer strictly necessary. 
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